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Abstract. Our concern in this paper is to describe the p-rank statification on the 
Siegel moduh space with Iwahori level structure over fields of positive characteristic. 
We calculate the dimension of the strata and describe the closure of a given stra- 
tum in terms of p-rank strata. We also examine the relationship between the p-rank 
stratification and the Kottwitz-Rapoport stratification. 



0. Introduction 

Fix a prime p, a positive integer g and an algebraic closure F of Fp. We denote by 
Ag the moduli space of principally polarized abelian varieties of dimension g over F and 
by Ai the moduli space of principally polarized abelian varieties of dimension g over F 
with Iwahori level structure at p. Let vr : Aj — > Ag be the canonical projection (see 
subsection 11.2.11 for details) . 

We denote by Ag^^ the subset of Ag of points that correspond to an abelian variety 
of p-rank d. Koblitz showed in his paper [7j that these sets form a stratification on Ag, 
more precisely: 

Theorem 0.1. Let d < g he a nonnegative integer. 
(1) A^f^ is locally closed 

(3) A^g"^ is equidimensional of codimension g — d 

Now consider the preimages A!"f^ := T^~'^{A^f^). These sets form a decomposition 

of Ai into locally closed subsets but the closure of A^f^ can in general not be written 

as union of sets A!'f\ We will nevertheless call them p-rank strata and refer to this 
decomposition as the p-rank stratification of Ai. In this paper we deduce statements on 
the p-rank stratification on Ai similar to those of Theorem 10. 1[ 



The dimension of the stratum corresponding to p-rank zero was already calculated 
by Gortz and Yu in section 8 of [5j. Our proof of the dimension formula for arbitrary 
p-rank (see below) is a generalization of theirs. 

Our approach is via the study of the Kottwitz-Rapoport stratification (KR stratifica- 
tion) on Ai , which is given by the relative position of the chain of de Rham cohomology 
groups and the chain of Hodge filtrations associated to a point of Aj. We use the result 
of Ngo and Genestier which states that the p-rank on a KR stratum is constant and 
thus the KR stratification is a refinement of the p-rank stratification. The KR strata 
are in canonical one-to-one correspondence with a subset Adm(/u) of the extended affine 
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Weyl group of GSp2g (defined in section [ITT]) . Most importantly the dimension, relative 
position and the p-rank on KR strata can be expressed in combinatorial or numerical 
terms on Adm(^) (for details, see section [TT^ . 

Denote the integer part of a number x by [xj . The following theorem summarizes the 
main results of this paper. 

Theorem 0.2. Let d < g be a nonnegative integer. Denote by M^*^) the subset of 
elements x € Adm(/i) which correspond to a KR stratum Ax which is top- dimensional 
inside A^f^ . 

(1) codim4'^) = L^ + ij 
(2a) If g — d is even, 

d'<d 

(2b) Ifg-d^l is odd, 

^=U4'^ U 

d'<d :reA/(d") 

d"<d, 2\g-d" 

(2c) Using the standard embedding W C S2g (cf. section [TTI]] . we have 

Ai^ ^ = -A-x- 

a;=t^Oii,gAdm(n) 
w({l....,a}) = {l,...,a} 

In contrast to the the p-ia.'nk strata on Ag^ the p-rank strata on Ai are in general not 
equidimensional. 

This paper is subdivided into three parts. In the first part we give some background. 
Here we give the definition of the extended affine Weyl group and a characterization 
of Adm(//) in section 11.11 and give the construction and required properties of the KR 
stratification in 11.21 The second part is the calculation of the dimension of the p-rank 
strata. Finally, we compare the KR stratification and the p-rank stratification in the 
third part, giving an explicit description of M*^"^) in section [3^2] and proving part (2) of 
Theorem [02] in [331 
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1. Background 

We keep the notation of the introduction. We also fix the following notation. For any 
set R and n-tuple v G i?" we denote the i-th component of v by v{i). We abbreviate 

a tuple of the form (ui, . . . , ui, f2, . . . , U2, • • • , t^m, • • • , Vm) by {v'^^^^ ■ ■ ■ Vm"'^) where ki 
denotes the multiplicity of Vi. If ki = 1 we will omit it. If R is an ordered set, e.g. 
i? = Z, and u,v i?" we write u < v iS u{i) < v{i) for every i = 1, . . . , n. 
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In order to simplify equations when using case analysis, we define for any statement 
P the term 

J. _ / if P is false 
^ I 1 if P is true. 

1.1. Preliminaries on GSp2p. 

1.1.1. Group theoretic notation. Let G be a reductive linear algebraic group over an 
algebraically closed field k, and let A be a maximal torus of G. These data give rise to 
a root datum {X* (A), Rq, X^{A), R'^) and its Wcyl group Wq = NqA/A. Wc denote 
by Qq its coroot lattice and by WaO = Qq ^ tlie affine Weyl group of G. For an 
element xq G Qq we denote by the corresponding element in WaO- The choice of a 
Borel subgroup containing A determines a set of positive roots Rq and a set of simple 
roots Ac. Associated to Aq we have the sets of simple reflections Sq = {sa, ct G Aq} 
and SaG = Sg^ • sg} of Wg and WaG where a denotes the (unique) highest root 

of Rq. Applying the standard identification of WaG with the set of alcoves on X^,{A)^, 
our choice of SaG places the base alcove in the anti-dominant chamber. We denote by £ 
and by < the length function and the Bruhat order on the Coxctcr system {WaG: SaG)- 
Whenever we deal with the case G = GSp2g (as it will be in the majority of cases), we 
drop the subscript G. 

Let GL2g denote the general linear group over F and D be its diagonal torus. We use 
the standard identification of the cocharacter group X^{D) with Z^^ and of WGL2g with 
the symmetric group S2g. 

Denote by GSp2g the group of similitudes corresponding to a 251-dimcnsional sym- 
plectic F- vector space {V,jp). We embed the GSp2g into GL2p by choosing a Darboux 
basis of V, i.e. a basis (ei, . . . , e2g) such that 

'4'{ei,e2g+i-i) = -ip{e2g+i-i,ei) = 1 

ioT 1 < i < g and il){ei,ej) = otherwise. Then the subgroup T C GSp2g of diagonal 
matrices is a maximal torus and the upper triangular matrices form a Borel subgroup 
B of GSp2^. 

T is the group of all elements t of the form diag(ti, . . . , t2g) such that there is a 
c{t) G with ti ■ t2g+i-i = c(t) for all i = 1, . . . , g. Hence the embedding of X^{T) into 
X^{D) = 1?^ yields the identificat ion 

X,(r) = {vG v{l) + v{2g) = v{2) + v{2g - 1) = ■ ■ ■ = v{g) + v{g + 1)}. 

We denote by e* G X* (T) the character which maps an element t G T to its i-th diagonal 
entry and c G X*{T) the character which maps t to its similitude factor. The positive 
roots in this setup are 

= i<i<j<g 

Plj = e* + e*-c l<i<j<g 
Pf = 2e*-c l<i<g. 

Then the simple roots are with 1 < i < g — I and Pg. 

The embedding of GSp2g into GL2g induces an embedding of the Weyl group W of 
GSp2g into = S2g. Then W is the ccntralizer of 

^=(1 2g){2 2g-l)---{g g + 1) 
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or equivalently the subset of all elements w which satisfy w{2g + 1 — i) = 2g-\-l — w{i) 
for all i. The simple (affine) reflections in this setup are 

.0 = 2g) 
Si,i+i = {i i + l){2g - i 2g - i + 1) 1 < i < - 1 
Sg = (5 5 + 1) 

1.1.2. The extended affine Weyl group. Using the same notation as above, we call Wg '■= 
X^{A) X Wg = NGA{k{{t)))/Alkltl) the extended affine Weyl group of G. Analogous to 
the case of the affine Weylgroup, we denote by the element of W which corresponds to 
the cocharacter xq E X^{T). Since every Wc-orbit of X^(T) is contained in a Qq coset, 
the affine Weyl group is a normal subgroup of Wq and we get a short exact sequence 

O^WaG^Wa^ X,{A)/Ql 0. 

Identifying i^" • w with the map x 1-^ w-x + xq, we consider Wg as subgroup of the group 
of affine transformations on X^:{A)^. Then the action of Wg on stabilizes the 

union of all affine hyperplanes corresponding to an affine root. Thus we get a transitive 
action of Wg on the set of alcoves. So the short exact sequence right-splits; Wg is the 
semidirect product WaG ^ where CIg is the subgroup of all elements which fix the 
base alcove. 

We define the length function and the Bruhat order on Wg as follows: Let x = ciwi 
and yi = C2W2 be two elements of Wg and Cj resp. Wi their Qg- and Wac-component. 
We say that x < y if ci = 02 and wi < W2 w.r.t. the Bruhat order on WaG- The length 
l{x) is defined to be (.{wi). 

Since we have W C Wqi^^^ and X*(T) C the extended affine Weyl group W 

of GSp2g is a subgroup of Wgl2<, • 

Now we recall the description of the extended affine Weyl group in terms of extended 
alcoves as in [6]. For this purpose denote by (ei)j=i_...^2g the family of standard basis 
vectors in I?9 and let 1 = (1, 1, . . . , 1) G I?^ . 

Definition 1.1. An extended alcove is a tuple of vectors x = (xq, . . . ■,X2g-i) in X^{D) 
such that for every i there is a € {1, . . . 2g} with 

where X2g := xq — 1. 

Then Wgi,^^ acts simply transitively on the set of extended alcoves. Using u = 
(wo, wi, W2> • • •) = (Oi — &2, ■ ■ •) as base point, we identify the extended affine 

Weyl group with this set. Using the same notation as in the definition, an extended 
alcove X corresponds to t^'^ ■ w € WGL2g • We call an extended alcove corresponding to 
an element of W a G-alcove. Note that an extended alcove x = [xq,... ,X2g-i) is a 
G-alcove if and only if there is a c € Z with 

Xi + 0{x2g~i) = c • 1 

for all i. 

Observe that since u lies in the closure of the base alcove, every extended alcove lies in 
the closure of the corresponding alcove of the Wa GL2g "Component. This observation and 
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the fact that the length function and the Bruhat order of the (affine, extended affine) 
Weyl group of GSp2g is inherited from the (affine, extended affine) Weyl group of GL2g 
(see [8], §4.1) imply that we can conclude the following lemmata from their analogue 
concerning the affine Weyl group. 

Lemma 1.2. Let a be an affine root ofGSp2g with corresponding wall Ha and reflection 

Sa- If X and y = Sa ■ x are two elements of W, we have x < y if and only if x lies on 
the same side of Hq as the base alcove. 

Lemma 1.3 (Iwahori-Matsumoto formula). The length of an element ofW equals the 
number of walls that separate the corresponding G-alcove from the base alcove, i.e. 

i{t^^-w)= Yl KA^o)l+ Yl K/5>^o) + i|. 

/3e-R+ /36-R+ 

The analogue of Lemma 11.31 is the usual Iwahori-Matsumoto formula for the affine 
Weyl group, the analogue of Lemma 11.21 is stated and proven in [8], Corollary 1.5. 

1.1.3. Minuscule G-alcoves. As we will see in the next section, the KR strata are in 
one-to-one correspondence with the minuscule G-alcoves of size g. In this subsection we 
give two useful characterizations of an element of W which corresponds to a minuscule 
alcove of size g. 

Definition 1.4. (1) We call a G-alcove x minuscule of size g if 

(1.1) ^^i 1^ Xi < Ui + 1 for all i 

(1.2) {xoii),xo{2g + l-i)} = {0,1} for ah i 

(2) An element x of is called /x-admissible for fj, £ X*(T) if there exists a w £ W 
such that X < w{fj,). We denote the set of ^u-admissible elements by Adm(;u). 

Remark. ()1.2p excludes merely the alcoves u and {ojo + 1, . . . ,^2^-1 + !)• 

Remark. It is obvious from the definition that there exists a (unique) element r E 
such that Adm{fj,) C W^r. It is given by 

, = i(0(^)l'^)).((l g+l)...(g 2g)). 

Lemma 1.5. If il.2\) holds, we can replace by 

(1-3) x,{^) = ^ ^ ^-l(^)<i 

where t^'" • w corresponds to x. 

Proof. Assume (jl.ip holds. Then 

w~^{i) > i xo{i) = a;„-i(i)-i(i) < w^„-i(i)„i(i) + 1 = 

w~'^{i) < i Xo{i) = a;^-i(i) + 1 > uj^-i(^i){i) + 1 = 1 
On the other hand if ()1.3p holds, we get 

Xkil) - ^k{i) = Xo{i) - (5^-l(j)<A: + k<k = 



1 if w^^{i) > i, i < k < w~^{i) 

iiw^^{i)>i, otherwise 

if w^^{i) < i, w^^{i) < k < i 

1 ifw~^{i)<i, otherwise 
XQ{i) if w~^{i) = i. 



6 



P. HAMACHER 



□ 



A more vivid description is given by a result of Kottwitz and Rapoport. For this let 
us denote = (l^s) O^^)) G X^T). 

Theorem 1.6 ([8], Theorem 4.5.3). A G-alcove x is minuscule of size g if and only if 
it corresponds to a ^-admissible element. 

1.2. The KR stratification on Ai. hi this section we recall the construction of the 
KR stratification and some geometric properties of KR strata resp. chains of abelian 
varieties corresponding to a point of a given KR stratum. 

1.2.1. Moduli spaces. Let > 3 be an integer coprime to p. The "classical" Siegel 
moduli problem associates the set of isomorphism classes {A,\,r]) to a locally Noetherian 
F-scheme S where 

• j4 is an abelian variety of relative dimension g over 5*, 

• \ : A — > A"^ is a principal polarization, 

• is a symplectic level- A^-structure on A. 

The moduli problem is solved by an irreducible quasi-projective F-scheme Ag of dimen- 

9(9+1) 
sion '^-^ — -. 

Now consider the functor associating the set of isomorphism classes of quadruples 
(A,, Ao, Ag, 77) to a locally Noetherian F-scheme S where 

• A, = {Aq Ai • • • j4g) is a sequence of abelian varieties over S of 
relative dimension g and the a are isogenics of degree p, 

• Ao,Ag are principal polarizations of Aq and Ag respectively, 

• 77 is a symplectic level- A^-structure on Aq, 

such that the composition of all arrows in the diagram 



An A^ . . . A 



^0 



(1.4) 



V 

1 
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equals multiplication by p. This functor is represented by a quasi-projective F-scheme 
Ai of dimension ^^^^^^ and the canonical projection ir : Ai ^ Ag, {A,, Xq, \g,rj) i-> 
(AQ,\o,rj) is a proper and surjective morphism. 

Let us abbreviate the notation of an S-point of Aj. We will usually denote it by A,. 
In the following we write A, = Aq ^ . . . ^ A2g for the sequence of abelian varieties of 
A, supplemented by its dual. This is, we identify Ag = A'^ via A^ and let ^2g-i := for 
i = 0, . . . , g. The morphisms ^2g-i ^2g-j+i are defined to be the dual of Ai. 

Given an abelian variety A, we denote by A\p\ the kernel of the multiplication by p. 
Recall that if A is defined over an algebraically closed field k of characteristic p, the 
group of j4[p](A;) is isomorphic to ('Z/p'ZY where < r < dim A is called the p-rank of 
A. 
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Since the preimage of a locally closed subset w.r.t. a continuous map is again locally 
closed, the p-rank strata on Aj are locally closed. We endow them with the reduced 
subscheme structure. 



1.2.2. Standard lattice chains and the first de Rham cohomology group. We begin with 
the definition of the standard lattice chains over Fjt]] and F. Let ei, . . . ,e2g denote the 
canonical basis of F((t))^^. The standard lattice chain over F[[t| 

A. = A_29 ^ A_2g+1 ^ . . . ^ Ao 

is given by A_j = (ei, . . . , e2g-i,t ■ e2g-i+i, ■ ■ .,t-e2g)vitj C ¥{{t)f9 where {ei, . . .,e2g} is 
the standard basis of F((t))^^ and (. . •)F[i] denotes the F[[f]]-submodule which is generated 
by the elements inside the brackets. Denote by ( , ) the bilinear form represented by the 
matrix J2g := anti-diag(— l^^-* l^^^). Obviously \-g-i is the dual of \-g+i w.r.t. t~'^ • ( , )• 
We denote by 

A, = A_2g A_2g+i ^ . . . ^ Ao 

the lattice chain obtained by base changing A, to F with respect to the evaluation at zero 
evQ : Fp]] — )• F. The images of the canonical bases of A_j w.r.t. the base change morphism 
are bases {e\, . . . , e|^} of A_j such that the linear map A_.j — )• A^j+i is represented by 
the matrix diag(l(23-i) q l(«-i)). We identify Ai_2g = A^^ for ah z = 0, . . . , g via the 
non-degenerate bilinear form Aj_2g x A_j — )• F given by the matrix J2g. In terms of this 
identification the morphism Aj_i_2g ^i-2g is the dual of A_j — )• A_j+i. Thus we can 
also write 

(1.5) A. = A^ ^ • • • ^ A^/i A_^ ^ • • • ^ Ao A^ 

For any F-algebra R we denote the base change of A, and A, to R resp. by 
resp. A,,ij. 

We denote by H^^^{A/S) or simply H^^{A) the de Rham cohomology of an 5-scheme 
A S. We are interested in the case where i = 1 and A is an abelian variety of 
relative dimension g. In this case IL^^{A/ S) is a locally free Os'-module of rank 2g 
and the Hodge - de Rham spectral sequence degenerates at Ei, yielding an inclusion 
OJA '■= o,*{^\/s) ^ Hqj^(^/S'). This embedding makes oja Zariski-locally a direct 
summand of rank g of Hqp^(A) ([T], §2.5). Furthermore, we have a natural isomorphism 
Rhni^VS) = iihni^/sr m §5.1). 

If 5 = Spec k is the spectrum of a perfect field of characteristic p, the Hodge-filtration 
of Hqp^(A/S') can also be given in terms of Dieudonne theory. For this we denote the 
(contravariant) Dieudonne module of a finite, commutative, p-torsion group scheme K 
over k by {0{K),F,V). 

Theorem 1.7 (|10j. Cor. 5.11). Let k be a perfect field and A an abelian variety over 
k. There is a natural isomorphism Hqj^(A/ Spec A;) = B(A[p]) which identifies loa with 
VB{A[p]). 
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Now let S = Spec R be affine and Noetherian. It is shown in de Jong's paper [4J that 
if we apply H^j^ to the diagram p.4|) we get a diagram of C^-modules 



H^r(^o) 



I9 



such that 

• The horizontal sequences are dual to each other. 

• qo and qg define non-degenerate alternating forms on Hqj^(Ao) and Yi\^Y{,{-^g) 
respectively. 

• Coker(H^j^(^j) HQj^(^j_i)) is a locally free i?-module of rank 1 for i = 
l,...,2g. 

• The composition of all morphisms in the diagram is zero. 

Any diagram having these properties is locally isomorphic to A,^/j. Furthermore ujaq 
and LOAg are totally isotropic w.r.t. the bilinear forms qo resp. qg (see [3]). 



1.2.3. The local model. Following the book of Rapoport and Zink ^llj, we obtain a local 
model diagram for Ai, 




ivrloc 

i.e. a diagram of F-schemes where is smooth, ip is smooth and surjective and Aj = M'pi 
etale locally. 

For any Noetherian F-algebra R, the i?-valued points of this diagram are given by 

AiiR) = {iA,,L,);A,eAiiR),i,:B},^{A,)^A.^R} 

{Mi : J--i ^ A-i,R is locally a direct summand of rank g. 
{^-i)i=0,...,2g] yi<2g : A_i_i ^ A_i maps T^i-i to 
Vi : J^-i,Ti-2g are in duality w.r.t. A^j = Ai_2c, 

ip{A,,u) = A, 

lp{A,,i,) = (ti(wAj)i=0,...,23- 



Here we mean by an isomorphism a tuple of i?-linear isomorphisms ij : Hpj^(^j) ^ 
A_j which commute with the canonical morphisms A_j_i — t- A_j resp. their dual for 
< i < (7 and identify the bilinear forms qo, qg with those of (jl.Sp up to a constant. 
Denote by Aut(A,) the group scheme over F whose i?-points are the automorphisms of 
A,^/j. Then ip is an Aut(A,)-torsor and is Aut(A,)-equivariant with respect to the 
canonical left action on Ai and Mj°|. 
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We have a bijection between ii-points of M'pi and diagrams of iijf]] -modules 



A-2g,_R ' > A_2g+1,/J ' > • • • ' > Ao,_R 



t ■ Ao,_R 



• X-i/L-i are projective i?- modules of rank g. 

• L_j and Li-2g are dual to each other with respect to the bilinear form t^^ ■ ( i )• 

Here a sequence J", € Mj°|(i?) corresponds to a diagram (jl.6p with L_j being the 
preimage of F-i w.r.t. the canonical projection \-i^R — )• X-i^n/t ■ X-i^R = A_j^/j. Using 
that projectivity is a local property, it is easy to see that this indeed defines a bijection of 
i?- valued points of M^"! and diagrams of this form. Obviously this bijection is functorial. 

1.2.4. Construction of the KR stratification. Here we recall the construction of the KR 
stratification of Ngo and Genestier in j9]. We denote by LGSp2g resp. L+GSp2g the loop 
group resp. the positive loop group of GSp2g. Let B be the standard Iwahori subgroup, 
i.e. the preimage of B w.r.t. the reduction map L+GSp2g — )• GSp2g and J? := LGSp2g /B 
the affine flag variety. 

Definition 1.8. Let i? be a Noetherian F-algebra. 

(1) A lattice in -R^F^ is a sub-^M-module L such that t^ Rltfa c L C t-^Rftfa 
for some N and such that t~^ Rft^/L is a projective i?- module. 

(2) A complete periodic lattice chain is a sequence of lattices {Li}i^z with C Li 
such that for every i we have that Li/Li^i is a locally free i?- module of rank 1 and 
Li+2g = t~^ ■ Li. We call {Li}i^x self-dual if Zariski-locally on R, there exits a unit 
c G R(it)) such that -L_j = c • L^, where denotes the dual of Li w.r.t. the bilinear 
form ( , ). 

We expand A, to a self-dual periodic lattice chain by setting Xi+2g-r '■= t^ ■ Xi. It is 
well-known that the map 

M{R) — )• {self-dual complete periodic lattice chains in R{{t))} 
g-B ^ g ■ X, 

is bijective i.e. the functor associating the set of self-dual complete periodic lattice chains 
in R([t))'^^ with R is represented by J?. By expanding the diagram ()1.6p we obtain an 
embedding Mj°| ^ M. We denote by Mx '■= BxB/B the Schubert cell associated to an 

element x G W. Then Mj"! can be written as disjoint union of Schubert cells. 
Proposition 1.9 (i9j Cor. 3.2). M^^l = UxeAdm^ -^4 



(L6) 

such that 



t-X- 



2g 



t ■ X^2g+l,B ^ 
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Remark. (1) Its left action identifies B witli the group of automorphisms of A, that fix 
( , ) up to a constant. Thus the Schubert cells in Mj°| coincide with the Aut(A,)-orbits. 

(2) Let X € W and x = (xq, . . . , X2g-i) the corresponding extended alcove. A short 
calculation shows that x ■ A_j = (t^2a-i(i)+i . g^.- j = . . . ^ '^9)¥lt} thus the corresponding 
subspace C A_j has basis {e*-; {x2g-i — uj2g~i){j) = 0}. 

Definition 1.10. For any x G Adm(/u), we define the KR stratum Ax = (p{ip~^ {J-(ix))- 

Now ip is Aut (A. )-equivariant . thus ijj~^(T£x) is Aut(A,)-stable. Since (p is a Aut(A,)- 
torsor, the property of being locally closed descends to Ax- We endow the KR strata 
with the reduced subscheme structure. The properties of KR strata can be deduced 
from the analogous properties of Schubert cells in a similar manner. 

Proposition 1.11 ([9], §4). Let x G Adm(/n). 

(1) Ax = \ly<x ■^y 

(2) Ax is smooth of pure dimension i{x) . 

In particular, the Ax form a stratification of Ai 

1.2.5. The p-rank on a KR stratum. From now on let k be an algebraically closed field. 
The calculation of the p-rank on a given KR stratum of Ngo and Genestier in [9], 
Thm. 4.1 also proves that x determines the kernels of the isogenies of the chains A, 
corresponding to an /c-point of Ax up to isomorphism. We give a proof quite similar to 
theirs using Dieudonne theory. 

Recall that up to isomorphism there are only three finite group schemes of order p 
over k: 

• TLjpTL^ the constant scheme. 

• Op, characterized by ap{R) = {u G R;uf = 0}. 

• fip, characterized by fJ.p{R) = {u & R;uP = 1}. 

The corresponding Dieudonne modules are characterized by 

• B(Z/pZ) ^ F, F bijective, ^ = 0. 

• B(Qp) F = V = 0. 

• B(^p) F = 0,V bijective. 

Proposition 1.12. Let A, G Ax{k). Denote the kernel of the isogeny vlj-i — > Ai by 
Ki. ~ 

(1) Ki = fj,p iffw{i) = i,Xo{i) = 1. 

(2) Ki ^ Z/pZ iffwii) = i,xoW = 0. 

(3) Ki^ap iffw{i)^i. 

Proof. It suffices to prove (1). Then (2) follows by duality and (3) by exclusion. Now 
the exact sequence of commutative, finite, p-torsion group schemes 

Q^Ki^Ai_i[p\ ^A,[p\ 

gives rise to an exact sequence of Dieudonne modules 
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Now /3 restricts to a surjection VB{Ai_i\p\) VB{Ki), thus we have FB(Aj„i[p]) ^ 
a(D(^i[p])) if and only if VlD){Ki) / 0, i.e. Ki ^ /ip. By Theorem O this translates to 
the equivalence 

Choose an isomorphism Hpp{(j4,) = A, and let {ej, . . . , el} be the bases we described 
in fTXm 

B{k) acts on Hj^p^(Aj_i) by multiplication on the left of block matrices of the form 

U2g+l-i 



M Ui.i 

where the Ur are upper triangular r x r matrices over k. Thus Q(H^j^(ylj)) = {e^j^^;j 7^ 
2g + 1 — i)f is jB(/c)-stable, so the condition ujAi_i ^ is invariant under 

the ;S(A;)-action. Let x = (xq, . . . xig-x) denote the extended alcove corresponding to x. 
Then we may assume that ujAi = (e^; {x^g-i — uJ2g-i)ij) = 0)^ (see remark in subsection 
[1231) • We get 

<J4> (x2g+i-j - u;2g+i-i)(2g + 1 - i) = 

^ X2g+i^i{2g + 1 - i) = -1 

<^ xo(25- + 1 - z) = and w-^{2g + l-i)<2g + l- i 

44> 2;o(«) = 1 and w^^i^i) > i. 

Now Lemma 11.51 implies that the last line is equivalent to xo(i) = l,w{i) = i. □ 

Corollary 1.13 ([S], Thm. 4.1). The KR stratification is a refinement of the strat- 
ification by p-rank. The p-rank on the stratum Ax,x G Adm(;u) is given by ^{i € 
{1, . . . ,g}; w{i) = i} where x = t^"w,w € W and where w is considered as an element 

0fS2gZ)W. 

Proof. Let A, € Ax{k). Since the multiplication by p is the composition of the maps 
Ai —7- its kernel is an extension of the Ki. Thus 

29-1 , 
logp(#^oN(A:)) = ^ogp{#Ki{k)) = logp(-#{i G {1, . . . , 25}; wii) = i}) 
i.=0 

which gives the result. □ 

The following result about KR strata allows us to calculate the dimension of A^f^ . 

Corollary 1.14. Denote by Adm{p,Y'^^ the set of fi- admissible elements, which give rise 
to a KR-stratum on which the p-rank is d. Then 

dim^^'^^ = max £(x) 

a;gAdm(/i)('') 

Proof. Since A^f^ is the finite union of locally closed Ax, we get 

dim^^^^ = max dim^^. = max ^(x) 

xGAdm(/^)('*) a;eAdm(/j)('') 

□ 
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We call the a KR-stratum top-dimensional if it has the same dimension as the p-rank 
stratum which contains it. Since the KR-strata are equidimensional this is equivalent 
to saying that all its irreducible components have maximal dimension in the p-rank 
stratum. We call the corresponding /i-admissible elements of maximal length. Note 
that by Corollary 11.141 an element x E Adm(ii)^'^^ is of maximal length if and only if 
i{x) = max^,gAdm(M)('') ^(^')- 

2. The Dimension of the p-rank strata 

2.1. Combinatorics of the symmetric group. In order to estimate the length of 
;U-admissible elements we need some results from the combinatorics of the symmetric 
group. The following definition will help us to express the length of certain /x-admissible 
elements. 

Definition 2.1. Let a G Sg. We define 

= e {l,...,5f ; i < J < < 

Ba = e{l,...,5}2; i< j = a(j) <fT«} 

Ca = e {1,...,5}^; i < j < < 

The following proposition is a reformulation of a result of Clarke, Steingrimsson and 
Zeng; using their notation it states that INV = INVmt ([3j) Prop. 9). 

Proposition 2.2. Let a £ Sg. Then 
e{a) = 2{A^ + A^-i + B,) + a + C^-i + #{r, i < a{i)} + #{r, a{a{i)) < a{i) < i} 

Proof. Since the notation in [3] is entirely different from ours, we give an elementary 
proof of the proposition. We will reduce the claim to the following lemma. 

Lemma 2.3 (|3j. Lemma 8). Let a £ Sg be a permutation. Write Oi = (T{i). Then 
i < j < ai,aj > i} = #{{i, j); Oi < aj < i, aj > j} 
#{{i,j);i<j<ai,aj<j} = j); ai < aj < i, aj < j} 

The first equation of this lemma is proven in [2], Lemma 3. Note that the second 
identity is a consequence of the first. 

Now let a € Sg. To shorten the notation we write Oj = cr{i). Writing £{a) as number 
of inversions, i.e £{a) = #{{i,j); i < j,(T{i) > we get 

e{a) - - A„-i - B„-i = #{{i,j);i<j,ai>aj,ai>i,aj<j} 

= #{{i,jy, i < j,ai > aj,ai > i,aj < j,j > Oi} 

j); i < j,ai > aj,ai > i,aj < j,j < Oj} 
= «j < di < j,ai > i} 

+#{{hj); i < j < ai,aj < j} 
= i<j< ai,aj > j} 

a-i < Oj < i,aj < j} 
= {A^ + C^ + #{i; i < aj) 

+{A^-i +B^ + C^-i + a^(i) < Oi < i}). 
Since B^j = B^-i, the assertion follows. □ 
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(2.1) iia) - 2{A„ + A,-. +Ba)>]^- a{i) + i}. 
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Proof. As a consequence of Lemma 12.21 we get 

£{a) - 2{A, + A,-i + B,) > i < a{i)}. 

Now the left hand side does not change if we replace a by a^^. Hence it is also greater 
or equal cr{i) < i}. Thus, 

£{a) - 2{A„ + ^,-1 +B.)>]^- i < aii)} + < i}) = ^ ■ ^i^) + 

which gives the desired result. □ 



2.2. Calculation of dim^j . As in the preceding sections, we consider W as subgroup 
of 52^. Note that every element of W is uniquely defined by its images of 1, . . . , (7. In 
particular, so are its fixed points. So whenever we speak of fixed points of a Weyl group 
element, we only mean those which are smaller or equal to g (unless stated otherwise). 

For any subset F C {1, 2, . . . , (7} let C be the subgroup of elements whose 
fixed points are exactly the elements of F . Denote by Adm(/x)-^ the preimage of 
under the canonical projection "pr : Adm(/i) W^t^^ ■ w ^ w. Then by Proposition 

Adm(;u)('^) = J Adm(/i)^. 

FC{l,....g}, 
#F=d 



We fix a non-negative integer d < g and a set F = {/i < /2 < ... < fd\ C 
{1, 2, . . . , fl'}. Using Lemma 11.51 we may index the preimage of any element in 
by {0, 1}*^. For every w G we have pr~^{w) = {t^^>-"-o ■ w; v e {0, l}'^} where 



>,o(^) 



if w~^{i) > i 

1 if w^^{i) < i 
v{j) if i = fj 

[ l-v{j) iii = 2g + l-fj 



For V € {0, 1} denote by Adm(//) the subset of Adm(//) of elements that are indexed 
by V. Then 

ve{o,i}'' 

and we obtain a one-to-one correspondence 



i.e. an element of Adm(;u)^'^ is uniquely determined by its Weyl group - component. 
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We fix a vector v E {0, 1}'^. To simplify the notation we establish the following 
notation: For integers 1 < < 5 let 

i < j or (i = j = fk and v{k) = 1) 
i> j or {i= j = fk and v{k) = 0) 
i = j and i ^ F 
i ^ j ov I ^ j 
i y j or i X j 

Remark. There is also a geometrical interpretation of fixing F and v. According to 
Proposition 11.121 this is the same as fixing the Ki up to isomorphism. 

We now identify W with F2 x Sg, an element a ^ Sg corresponds to a permutation 
w &W with w{i) = cr{i), i = 1, ... 5 and a vector u € ¥2 corresponds to an element w 
with w{i) = i if u{i) = and w{i) = 2g + 1 — i if u{i) = 1, i = 1, . . . , g. (Recall that 
the elements of W are uniquely determined by their restriction to {1, . . . , g}.) Denote 
by Sg the subset of all a ^ Sg for which there exists a u G F2 such that ua G . 
Obviously it is the set of all permutations which fix the elements of F . Let a G Sg . A 
vector u G F2 is called cr-admissible if ua G , which is equivalent to 

u{i) = Vz G F 
u(i) = 1 Vi : a{i) w i. 

So if i is a fixed point of a then u{i) has a preset value, but we can choose the other com- 
ponents freely. However, this changes when we consider the maximal elements among 
them. 

Lemma 2.5. Let a G Sg , u,u' G F2 a -admissible with u' < u (Imposing the canonical 
order on ¥2 = {0,1}). Then t^("'<')'"'"(M'cr) dominates t^("<^)'"'"(ncr) w.r.t. the Bruhat 
order. 



i^j :^ 

iy j -.^ 

i^j -.^ 

i4j 



Proof. By induction, it suffices to prove the assertion in the case when u and u' differ in 
exactly one, say the i-th, component. By the observation we made above, this implies 
that a{i) 7^ i. The rest of the proof is the same as the proof of Lemma 8.2 in [5]. □ 

Let us agree on the following convention. Whenever we write an element x £ W as 
a product like x = t^° ■ w or x = t^° ■ (ua), we assume that xq G X*(T), u G F2, cr G 5p 
and w G W. We call xq, w, u resp. a its X*(T)-, W-, F^- resp. 5g-component. 

As a consequence of Lemma 12.51 and the description of Adm(/x)^''' at the beginning 
of this section we get the following assertion. 

Corollary 2.6. Assume that ■ (ua) is a maximal element with respect to the Bruhat 
order in Adm(/u)^'''. Then for all 1 < i < g 



u(i 



xo{i) 



1 if a{i) i 

if a{i) 76 i 

1 iia-\i)'<i 
iia~^(i)yi 



More precisely any element t^°{ua) is dominated by t^'^{ua) where xq and u are given 
by above description. 
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Definition 2.7. We call a /^-admissible element possibly maximal if it satisfies the 
condition of Corollary I2.6[ 



The following proposition is the core of this paper, it enables us to calculate the 
dimension of the p-rank strata and to determine the top-dimensional KR-strata. 



Proposition 2.8. Let x = t^°{ua) be possibly maximal, 
we have 

9(9 + I) 



[1) For all a € Sg we have 



i(w) 



+ d-#{i;a{i)=i}-iia) 



+2(yl, + A,-i + f)e{l,...,g}xF;i<f< aii)}). 

(2) In particular, for all a 

< 

(3) LetF = {g-d+l,g-d+2,...g} and a = {1 2)(3 A) . . . {g - d - 1 g - d) if g - d 
is even resp. a = (1 2)(3 4) ... {g — d — 2 g — d — 1) if g — d is odd. Then 



i{x) 



g' + d 



Proof. (1) We calculate £{t^°{ua)) using Lemma [L3l Recall that 
l{w)= Yl K/3'^o)|+ Yl l(/3'^o> + l| 

In order to calculate the sum on the right hand side, we divide it up into ten smaller 
sums. In the process we distinguish between the cases 

(a) /3€{l3l.;l<i<j <9} 

(b) f3e{(3fy,l<t<j<g} 

(c) l3e{Pf;l<i<g} 

and the values of {u{i),u{j)} G F| when we consider the cases (a) and (b) resp. the 
value of u{i) G when we consider the case (c). We denote these sums by ^a,{u{i),u{j))j 

(a) Summands coming from pfj. Let 1 < i < j < g. We need to check when w~^f5jj is 
positive and calculate (/3?^-,xo). Using the description of positive roots given in section 
1.1 we get 

(2.2) w-'^Plj > ^ u{i) = and {u{j) = 1 or a'^ii) < a'^ij)) 



( 1 ifa-Hi) 
{Pij,xo) = xo{i) - xo{j) = < if cr-i(i) 

[ -1 if(7-l(i) 



-< «,cr"Hi) -< j or a-^{i) )^ i,(7-^{j) > j 
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We will give the calculation of (0,0) ™ f^l^ detail - the other sums are calculated 
analogously. 



^a,(0,0) 



l<i<j<9 \ ^ 
u(i) = u(j) = \ 



E 



l<i<j<9 \0-l(j)<0-l(j) 



E K4->^o)|+ E K/3i;-'^o) + l| 

(7-l(i)><T-Mi) 



Here the second equality holds because by definition of "possibly maximal", u{i) = 
is equivalent to a{i) 96 i and by (|2.2p we have equivalence between w~^/3lj G and 
cr^^(i) < cr^^(j) in the case u{i) = u(j) = 0. Our calculation of {f5jj,xo) implies that 
the first sum equals (we always sum over i,j G {1, ■ ■ ■ ,g} unless stated otherwise) 

(2.3) #{{i,jy,i<j,a'\i)^i,c7-\j)^j,a-\i)<c7-\j)} 

(2.4) +#{{ijy, i < y i,cT-\j) j,a-\i) < a-\j)} 

and that second sum equals 

(2.5) #{(i,j); i<j,c7-\i)^i,a-\j)^j,a-\i)>a-\j)} 

(2.6) +#{(i,j); ^<j>-i(i) ^i,a-i(j) ^j>-i(i) >c7-i(i)}. 

Using the same arguments we get 

(2.7) S,,(o,i) = #{(i,j); i<j,a~\{) -< i,a-Hj)-j} 

(2.8) S,,(i,o) = i^{ii,j);i<j,(T-Hi)^i,a-Hj)yj} 

(2.9) = #{(i,j); i <j>-i(i) «i,(7^i(j) 

Altogether, EjjGFa equals 

(2.10) #{(i,j); i<i,a-i(i) ^i,(j-i(j) ^j>-Hi) <^7-i(i)} 

(2.11) +#{{i,j);i<j,a-Hi)yi,a-Hj)^j,a-\r)<a-\j)} 

(2.12) +#{(i,i); fT~'(i) > <r'\j) ^ j > i} 

(2.13) +#{(i,i); ^''(j) < -< i < j}. 

Let us impose that until the end of this proof whenever we refer to a equation we only 
mean the right hand side. Then (f2TT)]l is the sum of ([23]), I^Ji), ([23]) and I^M) while 
(EUD, <^A2\i and (l2J3]l are just reformulations of ([231), ([23]) and ([22D, respectively. 

(b) Summands coming from f3fj . We proceed as in the first case. 

i(;"Vr^, >0 ^ n(i) = n(j) = or (n(i) = and CT"^(i) < cr~^(j)) 
or {u{j) = and cr^^ii) > (y^^{j)) 



{0ijiXQ) = xo(i)+xo(j)-l 



1 iia'^{i) ^i,a-^{j) 

if a-^{i) -< i,a-^{j) j or a-^{i) ^ i,a-i(i) ^ 3 
-1 ifa"Hi) 
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We conclude 

(2.14) Sfe,(o,o) = #{(^,J); i < j,c7'\i) -< h<y-\j) < j,a-\i) < a-\j)} 

(2.15) +#{{i,j);i<j,a-\i)^i,a-Hj)^j,a-\i)>a-\j)} 

(2.16) +#{{i,j);i<j,(7-\i)yi,a-Hj)^j,a-Hr)<a-\j)} 

(2.17) i < j,a-\i) y i,a-\j) y j,a-\i) > a-\j)} 

(2.18) Sfe,(o,i) = #{(ij);i<j>-i(i)^i,a-i(i)«j>-i(0<fj-i(j)} 

(2.19) Sfe,(i,o) = #{(i,j); i < j,c7-\i) « i,a-i(j) ^ j,a-i(i) < ^-^(i)} 
Sfe,{i,i) = 0. 

Altogether, Y.i,j&2^^<id) + ^b,{i,j)) equals 

(2.20) #{(^,j); ^<J>"'(i) <^T-'(j)} 

(2.21) +2 . #{(i,j); a-i(0 > (7-i(j) ^ j > i} 

(2.22) +2 . #{(i, j); a-^i) < -< i < j}. 

Here ^M) is the sum of (ICTIl . (IXTTTl . pTTID . (f^TTUD . (TOD and (f^ : is the 

sum of l^rn\i and (gTTD; and (^72^ is the sum of (imjl and pi^ . 

(c) Summands coming from f3f 

w'^Pf > n(i) = 



Thus we get 



c,0 = l(i) + C7 ^(i) ^i} 

0. 



^c,l 



So the final sum is 

iie'^w) = #{i-a{i)^i} 

+#{{id);i<j,cj-\i)<cj-\j)} 

+2.#{(i,j); a'^{i)>a-\j)y j>i} 
+2.#{(i,j); cj-\j)<a-^{i)^i<j} 
= 9 + d - a{i) = i} 

+ M-£(.) 

+2 • {A^ + {(i, A:); ^-^(i) < ^ < j>(A:) = 1}) 
+2 . (A,-i + {(i, fc); i < A < a-\i), v{k) = 0}) 

= ^-^^±^ + d-#{i-a{i)=i}-l{a) 

+2[A, + + k); i<fk< cT(i)}) 
where the sum is taken over j G {1, . . . , ^r} and k G {1, . . . , d}. 
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(2) The claim is a consequence of part (1) of the proposition and Coroharv 12.41 Since 
k); i< fk < cr(i)} < B„ we get 



< 



5(5 + 1) 



+ d 



1 

2 2 



#{i;cr(i) = i} 



{g - #{i- = i}) 



Now i{t^°w) is an integer, hence the claim follows. 
(3) We apply the formula of part (1). Now 



#{ii,k); i< fk< ct(«)} 
a{i) = 0} 






d + (5. 



Hence 



5(5+1) 



2\g-d 



2\g-d 



□ 

Remark. Let x = • be possibly maximal. Denote by x' the preimage of x w.r.t. the 
homomorphism W/GSp2g_2d ^ ^ induced by the embedding GS\>2g-2d ^ GSp2g which 
adds the fe-th rows and columns (O^'^"^) 1 O^^s-'^)) for all A: E F and 2^ + 1 - A: G F. This 
amounts to removing the /-th and 2g + 1 — /-th coordinates of the X*(r)-component 
and skipping the fixed points of w (here we also mean those which are greater than g). 
Proposition 12.81^ 1) implies that 



l{x) - i{x') 



5 + 1 
2 



d + 1 
2 



In particular the difference is independent of x. So the length of x does not depend on 
V and to some extent it also does not depend on F. 



As a consequence of Corollary II . 141 and part 2 and 3 of Proposition 12.81 we obtain the 
first main result of this paper, which is part (1) of Theorem lU. 21 in the introduction. 



Theorem 2.9. The dimension of Aj 



(d) 



IS 
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3. Comparison between the KR stratification and p-rank stratification 
We are going to prove the statement of Theorem I0.2r 2) next. Note that 

^= U A= U U-^^ 

j/GAdm(/i)(<*) j/GAdm{/i)('') ^<y 

where the last equahty holds because of Proposition 11.11( 1). So our strategy will be 
examining for every ^-admissible x the set of /i-admissible y which dominate x w.r.t. 
the Bruhat order, respectively the p-rank on the KR strata associated to these y. 

3.1. Going up in p-rank. In this section we prove some first lemmas concerning ele- 
ments dominating a given /x-admissible element x. The main part of this section is the 
construction of a set of examples, which we will use most of the time when we want to 
construct dominating elements of given p-rank. 

Since the p-rank is no longer fixed, we also cannot assume F and v to be fixed any 
longer. Thus we have to introduce some more notation. Denote the set of fixed points of 
the 14^-component of an element x € Adm(/i) by F{x). We also have to reformulate the 
notion of possibly maximal (cf. Definition 12. 7p . We call x = t^°{ua) G Adm(/i) possibly 
maximal if for every i = 1, . . . ,g 

u{i) = 1 ^ a{i) = i. 

It is easy to see that this definition is equivalent the definition we gave earlier. Thus 
we can still use Corollary 12.61 implying that every x G Adm(/i) is dominated by a 
possibly maximal element y which has the same S'g-component such that F(x) = F(y). 
In particular if x € Adm(^)('^) then we also have y S Adm(/i)^'^-*. 

Lemma 3.1. A^'^ C \Ja><d-^^f^ 

Proof. By Theorem lO.ll fl). the right hand side equals 7r^^(^^'^^) which is a closed set 
containing A^f''. □ 

Corollary 3.2. Let < d < d' < g, x e Adm(^)('^) and y G Adm{n)^'^'\ Then x < y if 
and only if x and y are related with respect to the Bruhat order. 

Proof. This is an easy consequence of Lemma l3. II and Proposition 11.1 iT l) . □ 

Definition 3.3. Let w = ua W and a = Z\ - ■ ■ Zn the decomposition into disjoint 
cycles (in the usual sense of the cycle decomposition in the symmetric group, including 
the cycles of order 1). We say that Z is a cycle of w if Z G {Zi, . . . , Z„} such that either 
ord Z > 2 or if ord Z = 1, i.e. Z = (i) where i is a fixed point of a, we have that i is not 
a fixed point of w. If x G Adm(/x) has ly-component w we also call Z a cycle of x. The 
set of cycles of x is denoted by Z(x). 

By construction Z(x) is uniquely determined by F(x) and the S'^-component a of x. 
The converse is also true: Since Z(x) determines the cyclic decomposition of o", it also 
determines a. Now an integer \ < i < g occurs in a (unique) cycle of x if and only if 
i F(x), i.e. F[x) is the set of all integers \ < i < g which do not occur in a cycle of 
Z(x). 
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As a consequence we may reformulate our conclusion of Corollary 12.61 as follows. For 
every ^-admissible x there exists a possibly maximal element y G Adm(/i) such that 
X < y and Z(x) = Z(y). 

Also note that our new description of F(x) yields that for any x G Adm(/i)('^) 

ord Z = g — d. 

We simplify the notation for the coming calculations. Denote by Si,Sij € W the 
reflections corresponding to the roots /3f resp. Pfj. With respect to the identification 
^ = Ff X 5g we have 

Si = (o^^-^ho^^?-*)) -Id 

Si J = (0(f)) .(.j) 

Lemma 3.4. Let x = t^°w = t^°{ua) G Adm(/i) he possibly maximal, Z = (ci ■ ■ ■ ci) (z 
Z(x). Then there exists a possibly maximal y € Adm(/i) with x < y satisfying the 
following additional assumption depending on Z. 

(1) If Old Z = 1 then 

F{y) = F(x)U{ci} 
Z(y) = Z(x)\{Z}. 

(2) If Old Z = 2 then 

F{y) = F(x)U{ci,C2} 
Z{y) = Z(x)\{Z} 

(3) // ord Z > 3 then there exists an integer 1 < k < I such that 

F(y) = F(x)U{cfe} 

Z{y) = Z(x) \ {Z} U {Z'} 

where Z' = {ci ■■■ Ck-i Ck+i ■■■ Cm). 

Proof. Surely the condition on F{y) follows from the condition on Z(y). Now observe 
that due to Corollary 13.21 we can replace the demand that y dominates x with respect 
to the Bruhat order by the weaker demand that they are related. For this it is sufficient 
that y = s ■ X or equivalently y = x ■ s where s is an affine reflection. 

In each of the three cases the fact that y is possibly maximal will be an easy conse- 
quence of the fact that x is possibly maximal; its verification will therefore be omitted. 

(1) Consider y = s^x. Since cr(ci) = ci,u{ci) = 1, we get that ci is a fixed point 
of y implying the condition on Z(y). Now by Theorem 11.61 an element of the extended 
affine Weyl group is ^u-admissible if and only if the corresponding G-alcove is minuscule. 
Therefore the criterion of Lemma 11.51 can also be used to check whether an element of 
W is ^-admissible. In our case the fact that x meets this criterion obviously implies 
that y satisfies it; thus y £ Adm(/x). 

(2) Let y = Sci,c2 " ^- We have o"(ci) = C2,cr{c2) = c\ (and thus u(ci) = m(c2) = 
since x is possibly maximal). Therefore we get that F(y) = F(x) U {ci,C2} and Z(?/) = 
Z(x) \ {Z}. Using the same argumentation as in (1), we see that the fact that y is 
/x-admissible follows from Lemma 11.51 
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(3) Since / > 3, there exists a "local minimum" Cfc_i > < Ck+i with Ck-i / c^+i 
where cq := Q and q+i := ci. If Cfc_i > c^+i let y = Sc^c^+iX, otherwise let y = xsc^^ck.^- 
One easily checks that the conditions on F(y) and Z(y) are met, so that we only have 
to prove that y is /x-admissible. 

In the case Ck-i > Ck+i let y = ty°w' . Note that since x is possibly maximal we have 
w~^{ci) = cr~^(cj) = Ci-i for all i. To prove that y is ^-admissible it suffices to check the 
criterion of Lemma [1.5 1 for the coordinates where the X=K(r)-component or the preimage 
of the PV^-component have changed, i.e. at and c^+i. Since is a fixed point of w' 
there is nothing to check here. Thus 

yo(cfc+i) = xo(cfc) = 

implies that y S Adm(/i). 

In the case Ck-i < Ck+i, we have y = t^^w' with the same w' as in the previous case. 
Thus it again suffices to check the criterion of Lemma [L5] at the coordinate Ck+i- Indeed, 

yo{ck+i) = xoick+i) = 1 

□ 

Corollary 3.5. Let x = t^°w = t^°{ua) be an element o/ Adm(/i)'^°'^ such that a is not 
a product of disjoint transpositions. Then there exists ay & Adm(//)('^+^) such that 
X < y with respect to the Bruhat order. 



Proof. Since x is dominated by possible maximal element with the same cycles, we may 
assume without loss of generality that x is possibly maximal. Since the assertion on a 
is equivalent to postulate that x at least one cycle of order unequal to 2, the assertion 
follows from Lemma 13.41 ^1) or 13.41 ^3). □ 



3.2. Top-dimensional KR strata. In this section we give an explicit description of 
the elements of maximal length, which by definition correspond to the top-dimensional 
KR strata. As an immediate result of the previous section we get: 

Lemma 3.6. Let x € Adm(^)('^) he of maximal length and a its Sg-component. Then 
X is obviously possibly maximal. 

(1) If g — d is even, a is the product of pairwise disjoint transpositions. 

(2) If g — d is odd, a is either the product of [^-^\ disjoint transpositions or the product 
of — Ij transpositions and a cycle of order 3 which are pairwise disjoint. 

Proof. (1) If a was not the product of pairwise disjoint transpositions. Corollary 
13.51 would give us a y € Adm(^)('^^"'^'' with £(y) > i{x). Contradiction to Theorem 12.91 

(2) Since g — d is odd we can either use Lemma 13.41 ^1) or 13.4( 3) on x. Using the 
dimension formula once again, we see that this gives us an element y € Adm{fi)^'^~^^^ 
of maximal length. Thus the S^-component of y is the product of pairwise disjoint 
transpositions. In the case where we used I3.4l fl) this implies that a is the product of 
[^^J pairwise disjoint transpositions, if we used 13.4( 3) a is a product of [^^J — 1 
transpositions and a cycle of order 3, which are pairwise disjoint. □ 
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So we may reduce our considerations to x € Adm(//) respectively a £ Sg which are of 
the form described in Lemma |3.6[ Next we check when the left hand side and the right 
hand side of (j2.1|) are (almost) equal. 

Lemma 3.7. If a £ Sg is an involution, equality occurs in h2. 1\) if and only if = 0. 

Proof. The proof is straightforward: 

e{a) - 2{A^ + A,-^ +B.)-^- #{r, a{i) + i} 

= e{a) - 2{A„ + A,-i + B^) - #{i; i < aii)} 
= C^ + C^-i +#{i; a^{i) <a{i) <i} 

Since a is an involution the last summand equals zero and C„~i = C^. Hence the right 
hand side is 2 ■ C^, which proves the assertion. □ 

Lemma 3.8. If a £ Sg is the disjoint product of some transpositions and a cycle of 
order 3, we get 

e{a) - 2{A^ + A^-i +B^) = ^- c7{i) + \ 
if and only if = C„-i = 0. 

Proof. One can prove this lemma the same way as Lemma 13.71 with only a slight differ- 
ence: Depending on whether the cycle of order 3 is increasing or decreasing, one of the 
terms i < a{i) < a'^{i)},#{i; cr'^{i) < a{i) < i} equals zero and the other equals 
one. Thus if we add the two equations 

i{a)-2{A^ + A,-i+B^)-^-#{i;i<a{i^} = + C^-i + #{i; a\i^ < a{i) < i} 

£{a-^)-2{A^ + A,-i+B^)-^-#{i;a{i) <i} = + C„-^ + #{i; i< aii) < a\i)} 
we get 

2e{a) - A{A^ + A,-i + B„) - a{i) ^ i] = 2C, + 2C,-i + 1 
and hence the desired result. □ 

Note that since the left hand side of (|2.ip is an integer and the fraction part of the 
right hand side is ^, they always differ by at least \. 

Remark. One can show that the converse of Lemma 13.71 and 13.81 is also true. If the left 
hand side and right hand side of ()2.ip are equal resp. differ by ^ then = C^-i = 
and a is an involution resp. the disjoint product of some transpositions and a cycle 
of order 3. Indeed, for any cycle do of order greater than two for which the sequence 
(sign(cTQ^^(l) — (TQ(l)))j is alternating we get Cq-q 7^ or C^-i / because otherwise 

the sequence (|crQ'*'"'^(l) — cjQ(l)|)j would either increase or decrease, contradicting the 
fact that it is cyclic. This proves the converse of Lemma 13.71 The argument remains 
true under the weaker assumption that (sign(crQ'''"^(l) — o"g(l)))°lf]^'^" is alternating after 
removing some consecutive elements of the same value (Here both terms "consecutive" 
and "alternating" refer to the notion that the ord cjo-th element of the sequence has the 
first element as its successor), which implies the converse of Lemma l3.8i 

Definition 3.9. Let a € be an involution and e € {1, . . . ,2g}. We say that a 
embraces e if there exists an i such that i < e < cr(i). 



THE p-RANK STRATIFICATION ON Ai 



23 



Combining our previous calculations from section [2] with the combinatorial lemmas 
we get a classification of the top-dimensional KR-strata. 

Proposition 3.10. LetO<d<g, x = ^'^{ua) € Adm(^)W. 

(1) If g — d is even, x is of maximal length if and only if it is possibly maximal, a is a 
product of disjoint transpositions and Co- = 0. 

(2) If g — d is odd, x is of maximal length if and only if it is possibly maximal and one 
of the following conditions on a holds true. 

(a) a is the product of [^^J disjoint transpositions such that Co- = and a does not 
embrace its (unique) fixed point e which is not contained in F(x) (i.e. u{e) = 1). 

(b) a is the product of \_^^\ — 1 transpositions and a cycle of order three which are 
pairwise disjoint such that = C„-i = 0. 

Proof. An element x G Adm(^)('^) is of maximal length if and only if it is possibly 
maximal and all inequalities we used to prove Proposition I2.8r 2) are in fact equalities 
for X, with an exception in the case where g — d is odd. Here we allow that exactly one 
of these equalities does not hold true, but its left hand side and right hand side differ 
only by | (since we actually estimated i{x) against = dim^^'^'' + ^). Now the 

mentioned equalities are 

(3.1) #{{iJ)G{l,...,g}xF{xy,i<f<a{i)} = 

(3.2) {!,..., 5}; a(0=i} = ^ 

(3.3) i{a) - 2{A„ + A^-i + B^) = ^#{i; a{i) ^ i}. 

Recall that we limited the possible values of an 5'g-component of an element of maximal 
length in Lemma 13.61 Thus it suffices to check the assertion in the following cases. 

(1) Let g — dhe even and a the product of transpositions. Since the fixed points 
of a are exactly the elements of F(x) the first two equations are fulfilled. By Lemma [3771 
the third equality is equivalent to Co = 0. 

(2) Let g — d he odd. We have to consider two cases: 

(a) a is the product of [^^J disjoint transpositions. In this case, we get 

i-#{ie{l,...,g}; aii) = i} = ^ + ^ 

Hence (j3.ip and (|3.3|) must hold true. Subtracting the left hand side from the right hand 
side, we see that (|3.1|) is equivalent to 

{i; i < e < a{i)} = 

i.e. a does not embrace e. We have already shown that ()3.3p is equivalent to Co = 

(b) a is the product of [^^J — 1 transpositions and a cycle of length 3 which are 
pairwise disjoint: Since the left hand side of ()3.3p is an integer but the right hand side 
has fraction part ^, they differ by at least ^. Thus x is of maximal length if and only if 
Ca = Co-i = (use Lemma [321) and x satisfies the equalities ()3.ip and ()3.3p . But 
those equalities follow from the fact that the fixed points of a are exactly the elements 
ofF(x). □ 
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We denote by M^'^^ C Adm(;u)'^'^) the subset of elements of maximal length. 

As an application of this result we can deduce an explicit formula for the number of 
top-dimensional irreducible components of A^f^ for d > 0. The remaining part of this 
section will not be needed in the sequel. 

We recall the following result of Gortz and Yu. 

Proposition 3.11 ([15], Thm. 7.4). If Ax is not contained in the supersingular locus, 
then the stratum Ax is irreducible. 

Now let d > 0. Then the proposition above says that Ax is connected for each 
Adm(/i)("')(;u). Thus, 

T^M*-"'^ = ^{ top-dimensional irreducible components oi A^f^^. 

Proposition 3.12. Let < d < g. We denote by Cn ■= • (^^) the n-th Catalan 
number. 

(1) If g — d is even, 

(2) If g — d is odd. 

Proof The claim is certainly true if g = d as M^^) = Adm(^)(^) = {t'^^^^^;w G W}, 
so we may assume that g — d > 0. In order to calculate ^M^'^\ we have to count the 
number of a £ Sg of the form described in Proposition I3.1UI As the factor counts 
the number of choices of the set of fixed points and 2"^ is the number of choices for the 
F2-component, we are reduced to the case d = 0. 

We denote for any positive integer m 

Km ■= W G S2m'i o" is a fixed point free involution, = 0}. 

Mapping an element a € to the the string with length 2m with i-th letter X if 
a{i) > i resp. Y if a{i) < i induces a bijection between Km and the Dyck-words of 
length 2m. Thus = Cm, proving (1). 

Now let g be odd. We denote by the set of all a G S2m+i such that a is the product 
of m disjoint transpositions, does not embrace its fixpoint and satisfies Ca = 0. Also, we 
denote by the set of all a G S2m+i such that a is the product of m — 1 transpositions 
and a cycle of order three which are pairwise disjoint such that = C^-i = 0. We 
write n := [|J. Then by Proposition 13. 101 

Now we have a bijection 

a I—)- a', 

where, denoting the fixed point of a by e, the permutation a' is given by 

{cr(i) iii^e,g + l 
g + 1 iii = e 
e ii i = g + 1 
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In particular, = C„+i. Now we have a bijection 

n-l 

^ \J{-l,l}xK,xK^_,_,x{l,...,2i + l} 

i=0 

a (Sct,CTi,(J2,Ci), 

where ci < C2 < C3 are the elements of the cycle of order 3 of a and 

-1 if cr(ci) = C3 



1 if cr(ci) = C2 

0"2 = 0-|(ci,C3)- 

As ci tells us where to reinsert cj2 into o"i, this map is indeed bijetive. Thus 

n— 1 

= Y,'^ ■ {2i + 1) ■ Ci ■ Cn-i 

1=0 
n 

= J];(2n + 2) • Ci • - (4n + 2) • • 

= (2n + 2) • Cn+i - (4n + 2) • 
= (2n + 2) • Cn+i - (n + 2) • 
= n • C„+i 
This finishes the proof of part (2). 



□ 



Corollary 3.13. Letd > 0. Then the number of top- dimensional irreducible components 
of A^f' equals 



g\ g-d+1 

O g-d+l 



if g — d is even and 

'^'''{dj 2 
if g — d is odd. 

Remark. One can also give a decription of the number of top-dimensional irreducible 
components of . If g is odd then according to Proposition 8.9 in [5] there is no 
supersingular KR stratum among the top-dimensional ones, so that the above formula 
still holds for d = 0. If (7 is even, this proposition tells us that there is exactly one 
supersingular top-dimensional KR stratum. A formula for its number of connected 
components is given in [6], Corollary 6.6. 

3.3. The relative position of p-rank strata. The aim of this section is to describe 
the closure of a p-rank stratum as a union of KR-strata. The main part of it will be to 
prove the following theorem: 

Theorem 3.14. LetO <d< g. 
(1) If g — d is even, 



Af = U A^f\ 

d'<d 
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(2) Ifg-d^l is odd, 



Af = u \ U 

d'<d :reM(d") 

d"<d, 2\g-d" 



(3) 4^-'^ = u A 

x — t^O (iio-)SAdm((j) 

Recall that we have shown that the closure of a p-rank stratum is again a union of 
KR strata at the beginning of section 13.11 

We begin with the stepwise proof of the first two parts of the theorem. Note that 
we already proved in Lemma 13.11 that the closure of a p-rank stratum and strata corre- 
sponding to a higher p-rank are disjoint. Now we determine the intersection of Aip and 
a!^^ ^\ which is the main ingredient of the proof of part (1) and (2). 

Proposition 3.15. Let 1 < d < g. 

(1) Ifg-dis even then Af^'^^ C A^^fK 

(2) If g — d is odd then 



AfnAt'^ = Af-'\ U A^. 

Proof. (1) Let g — d he even. We show that every KR-stratum with p-rank d — 1 is 
contained in the closure of a KR-stratum of p-rank d. 

Let X = t^°{ua) G Adm{fj,Y^~^\ Since g — d + 1 is odd, a cannot be the product of 
^~2^^ disjoint transpositions. So by Corollarv 13.51 there exists an y G Adm(^)^'^) such 
that X < y and thus Ax C Ay. 

(2) Let g — dhe odd. Then by Theorem 12.91 we have 

dim4''~^) = dim^'^) > dim (^\Af^ . 

Thus no top-dimensional KR stratum of A^*^"^^ can be contained in the closure of A^'^'' . 
Since A^f' is a union of KR strata, we conclude that it is disjoint to any top-dimensional 
stratum in A^^ ""^^ . 

So we are left to show that any non-top-dimensional KR stratum Ax C A^^ '^^ 
is contained in the closure of Ai'P . We prove this by constructing for every x € 
Adm(/i)(^^^) \ M^"'^-'^) a y G Adm(/x)(°') which dominates x w.r.t. the Bruhat order. 
Now let X = t^°{ua) G Adm(/i)('^~-'^) be not of maximal length. We distinguish between 
three different cases: 

(a) a is not an involution with d—\ fixed points. Then x is dominated by an element 
of Adm(/i)('^) by Corollarv 1331 



(b) a is an involution with d — 1 fixed points, > 0. Recall that every element 
of Adm(/i)*^°'~^) is dominated by a possibly maximal element of Adm(/i)^'^~^^ which has 
the same 5g-component. So we can assume without loss of generality that x is possibly 
maximal. Since C^- > 0, we find i,j such that i < j < a{i) < We write ai = a{i) 
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and ttj = (T{j). Let y' = t^'ow' = f^'oiu'a') be defined as Sj^a^ ■ x. We have to check the 
/Li-admissibihty criterion of Lemma 11.51 at the coordinates i,j,ai and aj. 

yo(i) = xo(i) = w'^^ii) = ai > i 

y'oij) = xo{(^i) = 1 w'-^ij) = i<j 

y'oiai) = xoij) = w'^^{ai) = aj > ai 

y'Q{aj) = xo{aj) = 1 w'-^{aj) = j < aj 

So y' is /u-admissible. Obviously w' has d — 1 fixed points and a' is not an invohition. 
Hence y' is dominated by some y G Adm(^)('^^ by Coroharv 13.51 We have x < y' hy 
Lemma 11.21 since 

Wia^^^o) = xo{j) - Xo{ai) = -1, 

so y also dominates x. 

(c) a is an involution with d— 1 fixed points, Co- = 0. Since x is not of maximal length, 
it is not possibly maximal. However, using Lemma [2. 5 1 we may (and will) assume there is 
only a single coordinate I < ci < g such that tt(ci) = 1 and a{ci) / ci. Let C2 = o-(ci). 
Then a interchanges ci and C2 and u{c2) = 0. Now y = Sci,c2 ' x is obviously contained 
in Adm(//)*^'^) and thus also x < y by Corollarv 13.21 □ 

In order to deduce a comparison between arbitrary p-rank strata from this proposition 
we need the following lemma. 



Lemma 3.16. If d ^ g — 1, the stratum A^f is contained in A^f\ 
Proof. Let x = t^'o(u(T) G Adm(;u)('^~2)_ 

assume without loss of generality that x is 
possibly maximal. If there exists a Z € Z(x) of order 2 we done by Lemma l3.4f 2). 

Assume none of the cycles has order two. Then we can apply Lemma [3.4r i) or l3.4r 3) to 
it. We get an element y' G Adm{fi)^'^~^^ which dominates x. If we applied Lemma [3.4r i). 
then Z(y') does not contain a cycle of order 2, if we applied Lemma I3.4f 3) there is at 
most one. So unless g — d = 1 we can apply Corollary [33] to y' to obtain a y € Adm(/i)('^) 
such that X < y' < y. □ 

Proof of Theorem \3. 14^ 1-2). Successive application of Lemma l3. 161 yields that 

(3.4) At^ C A^ 

for any d' < [|j . Now we combine this assertion with Proposition 13.151 



(1) Let g — d he even. Then Proposition 13.15) savs that 4"^-^^'-^) C Af'^ for any 
d' < L^J • But then ([33]) implies that 



Af' c A'; 



for any d' < d. Thus we have [Jd'<d-^^f^ ^ A\'^'^ . Since the converse inclusion was 
proven in Lemma l3. II this gives the desired result. 

(2) Let g — dhe odd. Then Proposition 13.151 states that 



Af'~'^ \ U ^f'^ 
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for every d' < [^^J- In combination with (|3.4|) this imphes that 

I I /i(rf') 



4^^) D U 4'^ \ U -^^ 

d'<d :ceA/(<*") 

d"<d, 2\g-d" 



Since we know by Lemma l3.ll that A^p C |J^/<^4'^\ the claim is reduced to the 
following lemma: 

Lemma 3.17. Let g — d even and u < g — d be an odd integer. Then none of the 
top- dimensional KR-strata Ax C A\^^ are contained in the closure of A^f^^\ 

Proof. Assume the contrary: Then there exist x G Adm(/x)(°') of maximal length and 
y € Adm(/i)('^"'"") with x < y, i.e. we can find reflections s^^\ . . . , s^^'^ such that y = 
s^'^) • . . . • s(^) • X and £{s^^+^^ ■ s^^^ ■ x) = ^(s(') • . . . • s^^) ■x) + l for every l<l<k. 
But with each of these reflections we gain at most two additional fixed points. Thus we 
get A; > ^ and 



g"^ + d u + l 



£{y) = e{x) + k> + 



g'^ + d + u 



+ 1, 



contradicting Theorem 12.91 □ 
This finishes the proof of Theorem 13. 14( 1-2) . □ 

In order to prove the third part, we need the following lemma. 

Lemma 3.18. Let w G W and t"'^^) = ' 2 ' ) 5(2) . gW . j a reduced decomposition. 

Then s'--'^ = sq or s^-^^ = Sg} = g. 

Proof. We denote 

a ■■= s^^^ = So or s^-^'^ = Sg} 

b ■■= s^^^ = Si,i+i for some i} 

Then a + b = l{t'"^^'^) = ^is+R, 

Recall the description of the Sj in subsection II. 1.11 (Left) multiplication with sq or 
Sg changes exactly one coordinate of the Fg-component, multiplication with Sj^j+i only 
permutes its coordinates; in particular it does not change the number of coordinates 
which are zero resp. one. Since the Fg-component of t^^^^ resp. r is (0^^^) resp. (1^^^) 
we get a > g. 

On the other hand, multiplication with sq or Sg does not change the S^-component 
and multiplication with Sj^j+i induces a multiplication with an adjacent transposition on 
the S'g-component. Since the iS^-component of t^^^^ resp. r is Id resp. (1 g){2 g — 1) ■ ■ ■ 
we get b > £{{1 g){2 5 — 1) • • • ) = , proving our claim. □ 

Proof of Theorem \3. 14^ 3). Recall that for x,y S W, we have x < y if and only if x can 
be realized as a subsequence of a reduced decomposition of y. We know that an element 
y = ty"- (uV) e Adm(/x) is contained in Adm(^)(9-^) if and only if u' = (O^*"^) 1 O^^s-O) 
for some i and a' = Id. Using Lemma 13.181 and the observations made in its proof, we 
see that this is equivalent to the claim that y can be realised by removing one sq or Sg 
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from a reduced decomposition of t'^^'^) for some w € W. Thus x = t^°(ua) G Adm(;u) 
is dominated by an element y G Adm(fi)^^~^^ if and only if x can be realized as a 
subsequence of a reduced decomposition of some t^^f^^ with at most g — 1 reflections 
equal to sq or Sg. In particular we get that u ^ 0. 

Now let X = t^" • (ua) with u ^ 0. By repeatedly applying Lemma 12.51 and Lemma 
I3.4f l). we may assume that u{io) = 1 for exactly one integer Iq. Then zq F(x), so it 
occurs in a unique cycle Z = (Iq ■ ■ ■ in) of x. We proceed by induction on ord Z. 

If oidZ = 1, then x is possibly maximal. By subsequently applying Lemma 13.4( 2-3) 
we get a possibly maximal element y that dominates x such that Z(y) = {(io)} and thus 
y G Adm(^)(s-i). 

If ordZ > 2, let x = f''> ■ w = t^'> ■ {ua) := Si^^i, ■ x. Then 

uU) - ^ ' ' + 



1 i = n 

F(i) = F(x)U{zo} 

Z(£) = Z(x) \ {Z} U {Z'} 

where Z' = (ii • • • in particular ordZ' = ordZ — 1. To prove that x is ;U-admissible, 
it suffices to check the criterion of Lemma 11.51 at the coordinate ii. Indeed, we have 
xo(ii) = xo{io) = 1 and w~^{ii) > g > h because of u{ii) = 1. Now x < x by Corollary 
3.21 Using the induction hypothesis we find that there exists a y G Adm{p)^^~^^ such 
that X <x < y. □ 
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